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Summary 


Using data strategically to guide decisions and actions can have a positive effect on edu- 
cation practices and processes. This facilitation guide shows education data teams how 
to move beyond simply reporting data to applying data to direct strategic action. Using 
guiding questions, suggested activities, and activity forms, this guide provides education 
data teams with a framework and the tools and vocabulary needed to support an informed 
conversation around the data they generate or acquire. The guide walks data teams 
through five key steps in using data for informed decisionmaking and strategic action: 
setting the stage, examining the data, understanding the findings, developing an action 
plan, and monitoring progress and measuring success. 
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Why this guide? 


All educators, whatever their role, use data to inform their decisions. For example, suppose 
that math scores in a school have been rising. A data team (box 1) looking at the math 
scores for all students in the school might say, “This is great! These students are clearly 
learning a lot.” Or they might say, “Overall, students appear to be doing well, but are 
there some children whose math skills haven’t improved?” In the first case the team might 
respond to the improved math scores by exploring whether the new after-school math 
tutoring program had contributed to the higher scores. In the second case the team might 
look at the scores for each grade and classroom to reveal any gaps in student gains and 
provide more support to lower performing students and their teachers. Both situations 
involve data-informed decisions. 

This guide provides grade-, school-, and state-level education data teams — composed of 
teachers, administrators, staff, and other stakeholders — with steps, sample questions, and 
resources for using data more systematically and rigorously in education decisionmaking. 
Data teams can print out the templates in this guide and use them to direct their own 
data-informed conversations. 

Data and data teams 

Using data to inform education policies and practices is becoming a greater priority for 
educators, spurred by the increased accountability requirements of the No Child Left 
Behind Act of 2001 and by a growing understanding of the importance of data in edu- 
cation decisionmaking. Additionally, states and other entities are simply collecting more 
data. Individual school and institutional datasets are being shared and, for that reason, 
are being structured more consistently. Computer systems and software allow for quicker 
and easier access to data. With the higher stakes associated with student performance, 
growing expectations of education institutions, and increased awareness of the power of 
data, education organizations are expected to be data-informed — making decisions and 
taking action based on the facts presented through data. 


This guide provides 
education data 
teams with steps, 
sample questions, 
and resources for 
using data more 
systematically 
and rigorously 
in education 
decisionmaking 


Data can enable policymakers to make objective decisions about education systems, 
provide states with information on program effectiveness, and give teachers and admin- 
istrators information on student learning to influence instruction, programming, and 


Box 1. What is a data team? 

Data teams are groups of individuals dedicated to data inquiry and the outcomes it supports. 
Data teams address challenges, field questions, and collect insights from the greater com- 
munity of organizations. These teams monitor student and school performance and efforts to 
improve performance within the organization. Data teams also have primary responsibility for 
organizing, analyzing, and coordinating data-informed responses. 

Some data teams are diverse — with representation from school-, district-, and state-level 
personnel, policymakers, researchers, data managers, and family and community members. 
Other data teams comprise individuals with similar roles, such as teachers of the same grade 
or subject or members of a school’s leadership or planning committee. The composition of 
data teams varies by the question of interest, but ideally all members enrich and strengthen 
both the process and the results of data-informed conversations. 
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professional development. Additionally, providing students with data on their progress 
can motivate and engage them in learning (Hamilton et al., 2009). As educators become 
comfortable with the processes described in this guide, they will be able to convene and 
lead their own data teams, supporting colleagues in data inquiry and building a commit- 
ted group of educators accustomed to using data to take action. Although forming a data 
team is not required in order to have a data-informed conversation, a collaborative team 
can identify the resources and personnel needed to collect and organize data and provide 
diverse perspectives on questions, analyses, and actions. 

What this guide does 

This guide aims to build capacity for using data more comprehensively and effectively in 
the Regional Educational Laboratory (REL) Pacific Region and elsewhere. It focuses on 
helping educators use data that have already been collected — through the National Center 
for Education Statistics, REL Pacific, departments and ministries of education, school data 
systems, analytical and simulation software provided with textbooks, longitudinal data 
systems, and other means. 

This guide describes five steps in data-informed conversations that lead to strategic 
decisionmaking and action: 

1. Setting the stage. What question is to be addressed in this data-informed conversation? 
What information is needed to answer the question? Is the information available? 

2. Examining the data. What patterns do the data reveal, or what “snapshot” observa- 
tions can be made about the question? 

3. Understanding the findings. What are the possible causes for the patterns? 

4- Developing an action plan. How can a data team create an effective plan for address- 
ing the issue? 

5. Monitoring progress and measuring success. How can a data team know whether 
progress is being made on the issue? 

Each step is described in more detail below. 

Step 1. Setting the stage 


This step identifies the question to be addressed, the information needed to answer it, and 
the feasibility of accessing the information. 

What is the question? 

Crafting clear research questions is crucial for enabling data-informed conversations. 
The data team starts by framing the “what” question as simply as possible. The question 
may be very broad to start with, as in “What do the data tell us about our middle school 
students’ math achievement level?” But while it is important to begin the conversation 
by identifying the issue, the question needs to be more specific (box 2). Narrowing the 
question makes identifying and interpreting data clearer and easier. 

Answering the “what” questions can lead to other types of questions, such as those that 
look at relationships between two variables (“Is there a relationship between middle school 
students’ gender and performance on standardized math achievement tests?”) or those that 


Crafting clear 
research questions 
is crucial 
for enabling 
data-informed 
conversations 
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Box 2. School-based scenario for identifying the “what” question 


A data team interested in exploring how middle school students are doing in math might start 
with the very general question of “What do the data reveal about the math achievement of our 
middle school students?” Through discussion, the team might narrow the question to “What 
percentage of middle school students are achieving at or above proficiency on standardized 
math achievement tests?” This question identifies a specific group (middle school students), 
measure (standardized math achievement test), and benchmark or evaluation method (at or 
above proficiency). The data team will be guided by this focused question in examining the 
data, understanding the findings, developing an action plan, and monitoring progress and mea- 
suring success. 


look at whether one variable influences another (“Does having an after-school tutoring 
program increase middle school math performance on standardized tests?”). 

Keeping long-term goals in mind helps ensure that current and short-term activities align 
with the larger mission. Defining clear short-term objectives that support long-term goals 
also contributes to a continuous data-informed conversation. Tracking long-term goals 
might also require data that are not currently being collected. Realizing that can initiate a 
discussion on future plans for data collection. 


Keeping long-term 
goals in mind helps 
ensure that current 
and short-term 
activities align with 
the larger mission 


What information is needed to answer the question? 

After clarifying the question of interest, the data team needs to identify sources of informa- 
tion to answer it. Answering the question posed in box 2 (“What do the data reveal about 
the math achievement of our middle school students?”) requires access to the results of stan- 
dardized math tests for middle school students. Some questions also require separating data 
into smaller groups of interest. For the question on the relationship between student gender 
and standardized math performance, data would need to be separated by gender. 

At this (or any) time, the initial question can be further narrowed or expanded, but if the 
research question changes, the data-use process will need to be revisited from the beginning. 


Is this information available? 


It is best to review data sources that have already been analyzed and reported or that exist 
in raw form, such as from departments of education, school-level records or databases, 
and regional educational laboratories — and to bring them all into the conversation. For 
example, some education agencies issue annual reports with information that has already 
been analyzed and reported for entire schools and districts. In that case further analysis 
might not be needed. But if analyses have not already been done, or have not been com- 
pleted in a way that answers the research question, looking for raw data is the next step. For 
the example in box 2 possible sources of standardized math test results include the Stanford 
Achievement Test 10th edition (SAT-10) or state or jurisdiction benchmark assessments. 

Having just one data source might not be enough for decisionmakers to have confidence in 
the results of the analysis. Having multiple sources of information lessens the likelihood of 
ending up without enough data to answer the question. 
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Common types of education data include demographic, perceptual, performance, and 
program data (table 1). The type needed depends on the question. 

Data can be collected at the national, state, regional, district, school, classroom, and student 
levels. Data can be gathered at single points in time (cross-section data) or at multiple 
points (longitudinal data) to show how the same student or groups of students performed 
over time. While cross-section data are useful for questions about what is happening now, 
only longitudinal data can provide a comprehensive look at trends and patterns. 


Sometimes the data are insufficient to answer the question. That in itself is an important 
finding. A lack of adequate or reliable data to address important questions can inform 
recommendations for future data collection, even if the questions cannot yet be answered. 

Guiding questions 

• What is the question to be addressed in this data-informed conversation? 

• What information is needed to answer the question? 

• Is this information available? 

Template 1 can be used to record questions and potential data sources for addressing the 
question. 


Sometimes 
the data are 
insufficient 
to answer the 
question; that 
in itself is an 
important finding 


Table 1. Common types of education data 

1 Type of data 

Description 

Examples I 

Demographic 

These data include descriptors of students, 
such as gender, ethnicity, and socioeconomic 
status, and descriptors of the organization, 
including enrollment and attendance. 

• Percentage of students in each ethnic 
category. 

■ Number of students who live within 
five miles of the school. 

• Average attendance rate for the 
school year. 

Perceptual 

These data provide information on how 
stakeholders feel or what they observe about 
the organization or its activities. They include 
stakeholder (student, parent, staff, community 
member, graduate) surveys or questionnaires 
and observations. 

• Parents’ perceptions of school quality. 

• Students’ opinions on the importance 
of education. 

■ Community members' thoughts about 
a new school calendar. 

Performance 

These data include information on how 
students are performing and on their education 
outcomes. The data include information on 
types of assessments, grades and grade 
point averages, graduation and dropout rates, 
mobility rates, suspensions and expulsions, 
remediation rates, college acceptance and 
attendance rates, and career readiness. 

• Percentage of students who scored 
proficient or above on the state 
standardized assessment. 

• Percentage of students who enroll in a 
four-year college after high school. 

■ Number of suspensions in the middle 
school over the last school year. 

Program 

These data include descriptive information 
on how education and related activities are 
conducted within the organization. They include 
the textbooks used, the levels of staffing or 
professional development at the school, the 
schedule of classes, curricular sequences, 
instruction strategies, the nature and frequency 
of assessments, extracurricular activities, and 
even the school setting. 

• Teacher-student ratio in the school 
district. 

■ Number of competitive sports offered 
in the high school. 

• Average number of years of 
experience for all elementary school 
teachers in the state. 

Source: Authors. 
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Template 1. Setting the stage: recording questions of interest and data sources 


What is the question? What information is needed? What information is available? 


1. 


2 . 


3 


4 . 


5 . 


Source: Authors. 
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Step 2. Examining the data 


Step 2 examines the data for patterns and initial observations and then explores any data 
limitations. 

Looking for patterns and making observations 

Looking for patterns in the data and making “snapshot” observations are a first step toward 
answering the question. For some questions data from one period (such as one test or one 
school year) may be sufficient. For example, if the question relates to math proficiency 
for the current school year, only one year of data is necessary. If the question relates to 
patterns of middle school math proficiency over time, several years of data are needed, and 
the data team will be looking for patterns across years. Only after examining the data do 
patterns become apparent, whether surprising (such as a steep increase in average reading 
scores in one year), expected (such as average daily student attendance dropping toward 
the end of the school year), or repeated (such as teachers missing certain items on a certi- 
fication exam each year). 

During this step it may be possible to begin characterizing findings from the data as 
strengths and challenges. Strengths are results in the data that indicate a success; challenges 
are results that indicate that something is blocking improvement or higher achievement. 

Strengths and challenges can be captured through observations that arise from examining 
the data. These observations should be: 

• Specific. Focus on only one item in the data. They are straightforward and clear 
because they involve single sources. 

• Factual. Incorporate only the data — no assumptions, inferences, or biases. 

• Related. Begin to answer the question. 

For example, while investigating middle school math achievement, a data team might make 
the following observation about a possible success: “Each year during 2009-12 students showed 
a 5 percentage point increase in average achievement scores between grades 6 and 7.” The fol- 
lowing observations might be made about a challenge: “While student math scores improved 
overall, the scores of male students showed a 2 percentage point decline between 2009 and 
2012.” These observations are not opinions but statements of facts observed in the data. 

Exploring data limitations 

Before beginning data analysis, it is important to discuss any limitations in the data, so 
that the findings are appropriate. In particular, the data team should consider whether dif- 
ferent sources of data can be compared, as some test data are not designed for comparison 
across grades and years. That is why the increase in math scores in the previous example 
should not be unequivocally considered a success — more information on the context is 
needed to know for sure. Similarly, data that are not the same (for example, SAT-10 scores 
and grade point averages) cannot be combined and analyzed together. Such data need to 
be viewed separately, as they convey different information or perspectives on the question. 

Additionally, while data can inform decisions and provide the first step in investigating 
relationships between programs and outcomes, establishing causality requires further 


Before beginning 
data analysis, 
it is important 
to discuss any 
limitations in 
the data 
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exploration of the initial observations and findings and rigorous analysis. Data may not 
have been collected or stored in a form that permits comparisons between groups or that 
supports the level of analysis needed to draw robust conclusions. In that case, the data 
team should look into conducting further research and evaluation studies, if feasible, to 
examine any causal relationships of interest. 

Guiding questions 

• What “snapshot” observations can be made about the question, or what patterns 
are evident? 

• Are these observations or patterns related to the original question? If so, how? 

• What strengths or challenges arise from the data? 

Template 2 can be used to track observations. Template 3 can be used to list strengths and 
challenges. 


Step 3. Understanding the findings 


Observations are the basis for identifying the driving factors that underlie the patterns, 
strengths, and challenges shown in the data and for informing future action plans. For 
example, the math results in the previous example could lead a school principal to say, 
“Now that we see there is a challenge in improving math scores for male students between 
grades 6 and 7, let’s figure out why.” 

A key step in understanding data results is to begin a discussion among the data team and 
interested colleagues about one or two key challenges. Understanding why challenges may 
be occurring can help address them and might affect the findings related to the original 
question. These driving factors will be the focus of the action plan for addressing the chah 
lenges and improving results. 

Choosing a key challenge 

A review of the data often reveals several challenges. The data team seldom has the time 
or resources to explore all the challenges they uncover, so they will have to focus on just 
a few. Some challenges will be more actionable than others and thus might have a higher 
or lower administrative priority. Actionable challenges have driving factors that educa- 
tors can address or influence directly. Other challenges might be more difficult to address 
immediately, such as social factors that influence students at home (such as lack of super- 
vision, low socioeconomic status, single-parent homes). When possible, data teams should 
focus on challenges that are actionable and have a high priority in the school or district; 
however, they should also keep in mind other short- and long-term objectives, even though 
they might not be immediately actionable. These objectives serve as a broader context for 
the team’s ultimate goal. 

Brainstorming possible driving factors for strengths and challenges 

One way to identify driving factors for a key challenge (and thus the extent to which it is 
actionable) is to ask a series of “why” questions followed by “because” responses (figure 1). 
The data team can begin to consider possible driving factors behind the challenge by 
understanding the facts and making educated guesses about the driving factors. As the 


While data can 
inform decisions 
and provide 
the first step in 
investigating 
relationships 
between programs 
and outcomes, 
establishing 
causality requires 
further exploration 
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Template 2. Examining the data: observation tracking 


Data source(s): 



Is this observation ... 

Observations 

Related 
to the 

Specific? Factual? question? 


1. 


2 . 


3. 


4. 


5. 


6 . 


7. 


8. 


9. 


10 . 


Source: Adapted from Nebraska Department of Education (2012). 


Template 3. Examining the data: strengths and challenges 


Observed strengths Observed challenges 


Example: Over the last three years overall math Example: Over the last three years math achievement 

achievement scores have improved between grades scores have declined for male students between 
6 and 7. grades 6 and 7. 


Source: Adapted from Mid-continent Research for Education and Learning (2010). 
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Figure 1. Identifying the driving factors 



Once the potential 
driving factors 
are identified, 
data teams should 
check that the data 
support the ideas 


Source: Authors. 


experts in the jurisdiction or school, data team members are the most familiar with the stu- 
dents, issues, and community, and their knowledge forms a foundation for brainstorming 
possible explanations. It is important to stick to the available facts. Opinions and attitudes 
can bias how data are interpreted, and data teams must remain as objective as possible in 
reviewing the data and observations and refrain from drawing conclusions based on per- 
sonal feelings or experiences. 

To understand the findings, data teams should revisit any data limitations. As discussed, 
data might provide an initial understanding of the questions but be unable to support any 
causal conclusions. Data teams must recognize any limitations and uncertainty in the find- 
ings as they explore them further. 

Once the potential driving factors are identified, data teams should check that the data 
support the ideas. In the example from figure 1 the data team would need to determine 
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whether the observations “Students spend less time on math topics each year” and “Stu- 
dents were learning different math content over the past two years” are correct by explor- 
ing whether adequate data are available on how much time middle school students have 
spent in math classes in the past four to five years. If the assumptions are supported by the 
data, the driving factor can remain. If they are not, the data team should consider other 
potential driving factors that are supported by the data. This process should be repeated as 
many times as needed to identify a strong set of potential driving factors. 


Guiding questions 

• What are the possible causes for the observed patterns? 

• What school-based factors might have contributed to the challenges identified in 
step 2? Why might these challenges exist? 

• Do the data support the identified factors? 

Template 4 can be used to identify driving factors. 

Step 4. Developing an action plan 


Action plans 
typically include 
short-term 
objectives and a 
long-term goal 


Step 4 involves developing an action plan to reduce challenges and promote successes. 
The first part of the plan is to identify the key stakeholders who will be involved. In the 
example, if the jurisdiction or school identifies the math achievement of male middle 
school students as a key challenge, many people will need to be involved to effectively 
address the issue. Without the support of key stakeholders, even the best plan is unlikely to 
succeed. Stakeholders might include school principals, curriculum and instruction special- 
ists, assessment staff, teachers, and parents. 

Setting goals and objectives 

Careful planning helps data teams take the right actions to tackle the challenges they 
have identified. Action plans typically include short-term objectives and a long-term goal. 
Short-term objectives allow for tracking progress and identifying steps toward a larger 
goal. Having a long-term goal makes clear what the plan hopes to accomplish in the end. 
Clarity and agreement on objectives and goals can be improved by following the structure 
of SMART (specific, measurable, attainable, relevant, timely) goals (table 2). 


Table 2. SMART goal criteria 

1 Goal 

Criterion jj 

pecific 

Setting specific goals helps identify the people involved, exactly what the goal seeks to 
accomplish, a location and timeframe, any requirements and constraints, and precise 
reasons why the goal should be accomplished. 

Measurable 

Measurable goals allow progress to be tracked and help identify steps toward achieving short- 
or long-term goals. Markers of a measureable goal include answers to several questions: how 
much, how many, by when, and how will the team know that the goal has been accomplished. 

Attainable 

To be attainable, goals must be reasonable for the data team to accomplish within the given 
timeframe and resource constraints, such as abilities, financial capacities, and attitudes. 

Relevant 

To be relevant, goals must have meaning to stakeholders. Relevance ensures that 
achieving the goal will make a substantial and positive change. 

imely 

All goals should include a timeframe. Having an end date confers a sense of urgency and 
motivates the team to work diligently toward achieving the goal. 

Source: Top Achievement (2012). 
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Template 4. Understanding results: driving factor identification 



Source: Authors. 
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Developing the plan 


An action plan can be created for achieving objectives and goals by brainstorming poten- 
tial strategies and actions to reach the goals. A strategy can be reviewed in terms of: 

• Time. Is it possible, given the data team’s time limits? 

• Resources. Is it possible, given the team’s resources and level of buy-in from key staff? 

• Relevance. Will it affect the top driving factor? Is it related to the goal? 

• Data availability. Are data available to inform and monitor the team’s action plan? 

Strategies and actions are the foundation of the plan. Once these are determined, neces- 
sary resources and a reasonable timeframe can be identified. 


An action plan may also include steps to explore initial findings through a more rigor- 
ous research study. Incorporating such studies into an action plan allows data teams to 
identify the resources and data collection efforts needed for the studies. Well-designed 
and well-conducted studies can provide information on whether an action or program (for 
example, an after-school tutoring program) is the cause of certain outcomes (for example, 
an increase in math scores in grade 7). Rigorous studies, with thoroughly examined find- 
ings, establish a solid foundation for decisions. 

Guiding questions 

• How can an effective plan be created to address the issue? 

• What should change? 

• When should the changes occur? 

• What resources, including personnel and time, will be needed to lessen each 
challenge? 

• What activities will contribute to reaching the goal? 

• Are the proposed activities possible using the available resources? 


An action plan 
can be created 
for achieving 
objectives 
and goals by 
brainstorming 
potential strategies 
and actions to 
reach the goals 


Template 5 can be used to set SMART goals. Template 6 can be used to organize the team 
for action. 


Step 5. Monitoring progress and measuring success 


The final step involves monitoring progress to keep the action plan on track and measur- 
ing success to determine whether the goal has been reached. 

Monitoring progress-how and why? 

Data team members should regularly revisit their action plan. If the plan gets off track, 
perhaps through dropped responsibilities, turnover of key staff, or unforeseen challeng- 
es, the team should troubleshoot to determine what to do to get the plan back on track. 
Team members should regularly check in with one another, identifying challenges, making 
changes as needed, and celebrating successes. 

Measuring success-how to know when the goal has been reached 

Assessing progress is essential in using data to achieve desired improvement. As an action 
plan is being implemented, it is important to assess whether its strategies are being properly 
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Template 5. Developing an action plan: setting SMART goals 

Goal 


Is this goal 
SMART? 

5 pecific 

What is the goal trying to accomplish? 


Measurable 

How will this goal be measured? 


Attainable 

Is the goal achievable, given the available time and resources? 

Yes/no 

Relevant 

Does the goal have meaning for those whose decisions it will influence? 

Yes/no 

imely 

What is the end date for the goal? 



What resources are required to pursue this goal? 


What important dates are associated with this goal? 


What milestones will be measured along the way? 


What activities will need to be completed to attain this goal? 


Notes 


Source: Adapted from Get Organized Wizard S.M.A.R.T. Goal Planner (http://www.getorganizedwizard.com/ 
community/personal-organizing-tips/600-how-set-smart-goals.html). 


Template 6. Developing an action plan: organizing the team for action 

Strategic statement: 



Action steps 

Resources 
What is needed to 

Deadline and responsibilities 

What will be done? 

do this step? 

Who will do it? By when? 


1. 


2 . 


3. 


4. 


5 . 


Source: Adapted from Nebraska Department of Education (2012). 
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executed and its goals are being achieved. Some action plans will be short term and lend 
themselves to quick evaluations using schoohlevel data systems; other action plans will 
be more long term and may require more time to implement (such as strategies relying on 
annual standardized test outcomes). 


When assessing the effectiveness of an action plan, the data team should collect the same 
type of data from the same data source used to identify the challenge or success. When 
data are from different sources, differences in how the data were collected could yield dif- 
ferent results. In the previous example the SAT-IO was the initial source of data on math 
achievement; if a different test is used as the main data source the following year, the 
results will not necessarily be related. 


Evaluation provides opportunities for team members and colleagues to observe the results 
of their efforts. When data team members see positive impacts on student achievement, 
that strengthens their motivation to continue their efforts. In addition, evaluating the 
effectiveness of the strategies and actions allows the team to determine what is working 
and what is not. If actions or strategies are not working, the team can adjust them. 


Evaluation provides 
opportunities for 
team members 
and colleagues to 
observe the results 
of their efforts 


Educators make decisions every day that affect the students in their classrooms and school 
systems. Using data to inform these decisions helps ensure that they are guided by avail- 
able information, are grounded in factual observations, and lead to better outcomes for all. 


Guiding questions 

• How can a data team know that progress is being made on addressing the issue? 

• Is a plan in place to monitor progress toward the goals? 
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Note 


The authors appreciate the support of the many people who shared their expertise, guid' 
ance, and reviews to make this guide possible. The authors gratefully acknowledge the 
contributions of Michael Siebersma and Cori Stott, who helped develop the Using Data 
to Guide Action for School Improvement guidebook released by the Nebraska Department 
of Education, and Ceri Dean and Danette Parsley, authors of Mid-continent Research for 
Education and Learning’s The Power of Data handbook. Material and templates from both 
documents were adapted for this guide. 
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